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ABSTRACT 

This paper introduces Importance-Performance (IP) 
analysis as a method to evaluate teaching effectiveness in a 
university outdoor program. Originally developed for use in the field 
of marketing, IP analysis is simple and easy to administer, and 
provides the instructor with a visual representation of what teaching 
attributes are important, how important each attribute is, and how 
well the instructor performed on each attribute. Implementing IP 
analysis requires four steps: developing a set of attributes that 
accurately describe and reflect the topic of study, presenting the 
attributes to respondents in questionnaire form that requires them to 
rate importance and teacher performance for each attribute, analyzing 
data for the importance and performance values of each attribute, and 
plotting each attribute on a four-section action grid according to 
its rated importance and teacher performance. In an application of 
this method, 35 attributes describing teaching effectiveness were 
generated from teacher evaluations at North Carolina State University 
and other institutions. Faculty review and feedback resulted in a 
final list of 23 attributes, which were presented in questionnaire 
form to 72 students in rock climbing courses. Importance and 
performance means were calculated and plotted on a grid. The 
instructor's strengths were identified as technical, safety, 
organizational, and communication skills, while some weaknesses were 
apparent in actual delivery of the activity. (SV) 
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Abstract 

This paper introduces the use of Importance-Performance Analysis as a 
method to evaluate teaching effectiveness in a university outdoor activity 
program. An important feature of this approach is that it examines both the 
importance and performance of selected attributes and presents them in an easy 
to interpret format. 

Introduction 

Evaluation is an important part of any program or service. It can be 
used to assess program goals and objectives, quality and improvement, gain 
support and credibility for programs and assist in the marketing of a program 
or product (Flor, 1995; Ford & Blanchard, 1993). To be effective, evaluation 
should in clude input from three sources: from professionals within the 
organization (management, staff), from professionals outside the organization 
(consultants and peers), and from those participating in a service or program 
provided by the organization (Mengak, Dottavio, & O 'Leary, 1986 ) . 

Importance-Performance (IP) is an evaluation technique originally 
developed for use in the field of marketing. It is based on the premise that 
evaluation or feedback be obtained from the consumer (or in this case, 
students or program participants) (Martilla & James, 1977). IP is simple and 
easy to administer, provides the instructor with a visual representation of 
what attributes are important, how important each attribute is, and how well 
the instructor performed on each of the attributes (Guadagnolo, 1985). 

Importance-Performance Analysis 

Implementing the IP analysis requires four steps. In the initial stage 
a set of attributes are developed to accurately describe and reflect the topic 
of the study. Following the identification of attributes, a questionnaire is 
developed and a survey conducted. The questionnaire requires that respondents 
be asked to rate the importance of a particular attribute and rate the 
performance of the instructor on the same attribute. A five-point Likert-type 
scale is used (l=low, 5=high) for both importance and performance (Mengak, et 
al. , 1985). This step is followed by analyzing data for the importance and 
performance values for each attribute. In the final step, data are presented 
by plotting each attribute on an action grid according to its perceived 
importance and performance (Figure 1). 

Methods 

Generating a list of attributes is an important part of the IP 
procedure. For the purpose of this paper, a list of attributes for providing 
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feedback on an instructor’s teaching effectiveness were developed by reviewing 
course and instructor evaluation forms used by a variety of academic 
departments at North Carolina State University. Teacher evaluation materials 
from other instutions were also reviewed for possible attributes. 

This procedure generated a list of 35 attributes that focused on 
instructor’s teaching effectiveness. This list was reviewed by a group of 
faculty members in the Department of Physical Education at North Carolina 
State University. Faculty members were instructed to review the original list 
of attributes and encouraged to add or delete any attributes they felt were 
missing or irrelevant. Feedback from this group resulted in a final list of 
23 attributes (Table 1). 

Figure 1. Action Grid 
(Martilla & James, 1977) 
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Subjects 

A convenience sample of students enrolled in the researcher's basic rock 
climbing and intermediate rock climbing courses during the spring semester 
1995 were asked to volunteer as subjects (N = 72) in this study. Those 
students who volunteered to participate in the study were given a 
questionnaire to complete during the final class meeting. 
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Table 1. Instructor’s Teaching Attributes 
(N = 72) 



Code* Teaching Attributes 

A Demonstrates knowledge of the subject 
B Use examples to facilitate learning 
C Is highly skilled 
D Is able to demonstrate skills 

E Appropriate material for student’s skill level 
F Ensures safety of all students 
G Uses a variety of teaching techniques 
H Reviews previous class material 
I Is accessible to students 
j Starts and ends class on time 
K Enthusiasm for the material being taught 
L Uses fair evaluation methods 
M Knows students by name 

N Provides opportunities to practice skills 
0 Provides a challenging course 
P Presents material in a logical manner 
Q Pace of instruction is adequate 
R Summarizes material when appropriate 
S Stimulates my interest in the subject matter 
T Gives feedback to improve skills 
U Has a good rapport with students 
V Communicates subject material effectively 
W Recognizes good student performance 



Importance 

Mean 

5.0 

5.0 

4.8 
5.0 

4.9 
5.0 

4.3 
4.8 
4.6 

4.6 

4.4 
5.0 
4.8 
5.0 
4.8 
4.8 

4.7 

4.5 
5.0 
5.0 
5.0 
5.0 

4.8 



Performance 

Mean 

4.9 

4.4 

4.7 

4.6 
4.4 

4.4 

3.8 
4.3 
4.2 
4.0 

4.2 

4.7 

4.6 

4.5 

4.3 

4.2 

4.3 

4.4 

4.5 

4.3 

4.7 
4.5 

4.4 



♦Codes correspond to points plotted on Figure 2. 



Analysis of Data 

Analysis of data involved calculating the means of the perceived 
importance and performance attributes. Means were then matched and plotted on 
an action grid (Figure 2). The x and y axis were positioned on the action 
grid at the overall mean for each attribute (4.8 for Importance, 4.3 for 
Performance) This method was adapted from research conducted by Hollenhorst, 
Olsen, and Fortney (1992) who noted that most attributes tend to be rated high 
by respondents and therefore very little information is provided relative to 
problem areas or areas requiring attention. By incorporating the IP analysis, 
the instructor was able to view the importance and performance of a given 
attribute and its relationship to all others. 

Results 

The results of this study suggested that students were generally 
satisfied with the teaching effectiveness of the instructor. This finding is 
supported by the high importance and performance ratings noted in the quadrant 
"keep up the good work" (Figure 2). Students were particularly satisfied with 
the instructor’s knowledge of the subject matter, ability to demonstrate the 
skills associated with the activity, rapport with students, use of fair 
evaluation methods, opportunity to practice skills, stimulating interest in 
the activity being taught, communicating subject matter, ensuring safety, 
providing feedback to improve skills, and the material being taught is 
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appropriate for the student's skill level, in addition, the importance scores 
of six attributes fell on the overall importance mean (4.8). This finding 
suggests that the instructor is doing well in these areas and should continue. 



Figure 2. Action Grid for Teaching Attributes 
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Four of the attributes; is highly skilled, reviews class material, knows 
students by name, and recognizes good student performance were identified as 
high in performance, suggesting that students were satisfied with the 
instructor's performance and that the instructor continue to stay focused on 
these attributes. Two attributes, pace of instruction is adequate and 
material presented in a logical, orderly sequence were scored low on 
performance, in this situation the instructor may consider channeling more 
attention into the planning and delivery of course material. 

Respondents also identified five items that were low on both importance 
and performance, thus indicating these attributes as low priority. These 
attributes included: access to students, beginning and ending class on time, 
enthusiasm, and using a variety of teaching techniques. One item, summarizing 
class material was noted by respondents as performing well but not being that 
important, suggesting that less attention be focused in this area. 
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Discussion 

I-P analysis is an adaptable and easily interpreted technique that is easy 
to administer, score, and evaluate. Importance-Performance analysis may be a 
valuable tool in assessing the teaching effectiveness of instructors in 
university sponsored outdoor activity classes. Through this process, both 
instructional strengths and weaknesses can be identified. In this study, the 
instructor’s strengths centered primarily around technical, safety and 
teachinq skills, and the instructor’s organizational and communication skills, 
weaknesses might be in the actual delivery of the activity (challenging 

C ° UrSe «her S usefori-P “arsis' may include developing a set o£ attributes to 
assist program managers and field staff identify strengths and weaknesses of 
soecific outdoor programs or program components, evaluating student s or staff 
in leadership roles, or including IP as one part of a more comprehensive 
evaluation process. When used as a repeated measure, I-P analysis mayhelp 
detect changes in one’s teaching effectiveness, program delivery, or other 
ore-determined factors. In conclusion, Importance-Performance analysis can 
provide a useful and adaptable alternative to traditional evaluation 
procedures . 
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